Web Survey Bibliography
In the literature on questionnaire design and survey methodology, pre-testing is mentioned as a way to evaluate questionnaires (i.e. investigate whether they work as intended) and control for measurement errors (i.e. assess data quality). As the American Statistical Association puts it (ASA, 1999, p. 11): “The questionnaire designer must understand the need to pretest, pretest, and then pretest some more.” Clark and Schober (1992, p. 29) indicate why this need to pre-test: “Surveyors cannot possibly write perfect questions, self-evident to each respondent, that never need clarification. And because they cannot, the answers will often be surprising.”
In this Ph.D. thesis I have tried to systematically describe my experiences with pre-test research at the Questionnaire Laboratory at Statistics Netherlands, a cognitive laboratory which started its work in 1992. This text is not aimed at a theoretical discussion of cognitive laboratory methods, but focuses on the application of these methods: setting-up and carrying-out pre-test research, analysing the data and presentation of the results.
The thesis starts with an introduction of cognitive laboratory research, including the history of the CASM (Cognitive Aspects of Survey Methodology) movement and the history of the Questionnaire Laboratory at Statistics Netherlands. The next two chapters address aspects of computer-assisted interviewing. Since at Statistics Netherlands most social-survey questionnaires are computerised, this sets the conditions for pre-test research at the Questionnaire Laboratory. Chapter 2 discusses computer-assisted interviewing in general; chapter 3 addresses the effects of computer-assisted interviewing on data quality. These three chapters are introductory to the chapters that follow, the actual core of the thesis, in which the application of cognitive laboratory methods, including several case studies, are discussed.
The methods used at the Questionnaire Laboratory at Statistics Netherlands are discussed in chapters 4 and 5. Chapter 4 presents an overview of pre-test methods. Expert appraisal, focus groups, in-depth interviewing (including follow-up probing, meaning-oriented probing, paraphrasing, targeted test questions, and vignettes), and behavioural coding are discussed from a practical point of view, i.e. how they are applied in the Questionnaire Laboratory. Computer-Assisted Qualitative Interviewing (CAQI) is discussed in chapter 5. The CAQI method has been developed at the Questionnaire Laboratory to pre-test computerised questionnaires. With CAQI a pre-test protocol is integrated in a computerised questionnaire to be tested.
The next four chapters present case studies of cognitive research in which the methods addressed in the chapter 4 and 5 have been applied. These chapters discuss the design and the results (i.e. identified problems in the questionnaire and recommendations for improvement) of these studies.
Chapter 10 concludes this thesis with a summary. A number of identified problems in the investigated questionnaires are: vague or unclear wording, complex syntax, long question, double-barrelled question, conflict with previous question(s), question asks for specific information that is not available by heart, difficult to come to an answer because of complex calculation, overlapping or missing response items. In the conclusion, the identified problems are related to design errors in the questionnaire.
igitur Homepage (abstract) / (full text)
Web survey bibliography - Thesis, diplomas (29)
- A Meta-Analysis of the Effects of Incentives on Response Rate in Online Survey Studies; 2017; Mohammad Asire, A.
- Designing web surveys for the multi-device internet; 2015; de Bruijne, M.
- Rating Scales in Web Surveys: A Test of New Drag-and-Drop Rating Procedures; 2015; Kunz, T.
- Mixed-method feasibility study comparing the outpatient assessment of burn patients using a tablet device...; 2015; Mitchell, S. S.
- Facebook, Twitter, & Qr Codes: An Exploratory Trial Examining The Feasibility Of Social Media Mechanisms...; 2014; Gu, L. L.
- Open-ended questions in Web Surveys-Using visual and adaptive questionnaire design to improve narrative...; 2014; Emde, M.
- Design and Implementation of an Online Questionnaire Tool; 2014; Schaniel, R.
- User Modeling via Machine Learning and Rule-Based Reasoning to Understand and Predict Errors in Survey...; 2013; Stuart, L. C.
- Investigation of background acoustical effect on online surveys: A case study of a farmers' market...; 2013; Tang, Xi.
- Developing a New Mixed-Mode Methodology For a Provincial Park Camper Survey in British Columbia; 2013; Dyck, B. W.
- Classifying Mouse Movements and Providing Help in Web Surveys; 2013; Horwitz, R.
- Satisficing in Web Surveys: Implications for Data Quality and Strategies for Reduction; 2013; Zhang, Che.
- “I think I know what you did last summer” Improving data quality in panel surveys; 2012; Lugtig, P. J.
- Analyzing Functionalities for Online Questionnaire System (OQS); 2012; Atown, H. Y.
- Web panels in Slovenia; 2011; Lenar, J.
- Clarifying Survey Questions; 2011; Redline, C. D.
- Nonresponse and Measurement Error in Mobile Phone Surveys ; 2010; Kennedy, C.
- Internet-Based Measurement With Visual Analogue Scales: An Experimental Investigation; 2010; Funke, F.
- Social Networking Sites: Evaluating and Investigating their use in Academic Research; 2010; Redmond, F.
- E-epidemiology : Adapting epidemiological methods for the 21st century; 2009; Bexelius, C.
- Visual Design Effects on Respondents’ Behavior in Web-Surveys; 2009; Greinoecker, A.
- Improving survey response in mail and internet general public surveys using address-based sampling and...; 2009; Messer, B. L.
- Design Variations in Adaptive Web Sampling; 2008; Vincent, K. S.
- Internet-based survey design for university web sites : a case study of a Thai university ; 2007; Vate-U-Lan, P.
- On the cost-efficiency of probability sampling based mail surveys with a Web response option; 2005; Werner, P.
- Cognitive Laboratory Experiences : On Pre-testing Computerised Questionnaires; 2002; Snijkers, G.
- (Non)Response bei Web-Befragungen; 2002; Bosnjak, M.
- Web survey errors; 2001; Lozar Manfreda, K.
- A study of factors affecting responses in electronic mail surveys; 1997; Good, K. P.